High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence

Given i.i.d. observations of a random vector X ∈ R, we study the problem of estimating both its covariance matrix Σ∗, and its inverse covariance or concentration matrix Θ∗ = (Σ). We estimate Θ∗ by minimizing an l1-penalized log-determinant Bregman divergence; in the multivariate Gaussian case, this approach corresponds to l1-penalized maximum likelihood, and the structure of Θ∗ is specified by the graph of an associated Gaussian Markov random field. We analyze the performance of this estimator under high-dimensional scaling, in which the number of nodes in the graph p, the number of edges s and the maximum node degree d, are allowed to grow as a function of the sample size n. In addition to the parameters (p, s, d), our analysis identifies other key quantities that control rates: (a) the l∞-operator norm of the true covariance matrix Σ∗; and (b) the l∞ operator norm of the sub-matrix Γ∗SS , where S indexes the graph edges, and Γ∗ = (Θ) ⊗ (Θ); and (c) a mutual incoherence or irrepresentability measure on the matrix Γ∗ and (d) the rate of decay 1/f(n, δ) on the probabilities {|b Σij − Σ∗ij | > δ}, where b Σ is the sample covariance based on n samples. Our first result establishes consistency of our estimate b Θ in the elementwise maximum-norm. This in turn allows us to derive convergence rates in Frobenius and spectral norms, with improvements upon existing results for graphs with maximum node degrees d = o( √ s). In our second result, we show that with probability converging to one, the estimate b Θ correctly specifies the zero pattern of the concentration matrix Θ∗. We illustrate our theoretical results via simulations for various graphs and problem parameters, showing good correspondences between the theoretical predictions and behavior in simulations.